Statistical Evaluation Metrics

📐 What Is a Confusion Matrix?

A Confusion Matrix is a table used to visualize the performance of a classification model. For binary classification it is a 2×2 matrix with four key components:

📏 Definition & Formula

Accuracy is the ratio of correct predictions (both positive and negative) over the total number of predictions.

🎯 Definition & Formula

Precision is the fraction of positive predictions that are actually correct. It answers: "Of everything I labeled positive, how much was truly positive?"

🔍 Definition & Formula

Recall (also called Sensitivity or True Positive Rate) is the fraction of actual positives that the model correctly identified. It answers: "Of all true positives, how many did I catch?"

⚖️ Definition & Formula

The F1 Score is the harmonic mean of Precision and Recall. The harmonic mean is used instead of the arithmetic mean because it penalizes extreme differences between the two values more heavily.

📊 F-Beta Score — Weighted Variant

When you want to weight Precision or Recall differently, use the F-Beta Score:

🛡️ Definition & Formula

Specificity (True Negative Rate) is the proportion of actual negatives correctly identified by the model. It is the counterpart of Recall for the negative class.

📈 What Is the ROC Curve?

The ROC (Receiver Operating Characteristic) curve plots the True Positive Rate (Recall) against the False Positive Rate (1 − Specificity) at various classification thresholds.

🏆 AUC (Area Under the Curve)

AUC is the area under the ROC curve. It ranges from 0 to 1 and measures the model's overall ability to distinguish between classes.

📉 Definition & Formula

Log Loss (Binary Cross-Entropy) measures how well the model's predicted probabilities match the true labels. Lower Log Loss = better model.

Where $y_i$ is the actual label (0 or 1), $\hat{y}_i$ is the predicted probability, and $N$ is the total number of samples.

📐 MAE — Mean Absolute Error

The average of absolute differences between predicted and actual values. Robust to outliers.

📊 MSE — Mean Squared Error

The average of squared differences between predicted and actual values. Penalizes larger errors more heavily.

📏 RMSE — Root Mean Squared Error

The square root of MSE, bringing the error back to the original scale. The most widely used regression metric.

🎯 R² — Coefficient of Determination

Indicates how much of the variance in the dependent variable is explained by the model. Closer to 1 = better fit.

🐍 Classification Metrics with Scikit-learn

from sklearn.metrics import (
    accuracy_score, precision_score, recall_score,
    f1_score, confusion_matrix, classification_report,
    roc_auc_score, log_loss
)
import numpy as np

# Ground truth and predicted labels
y_true = np.array([1, 0, 1, 1, 0, 1, 0, 0, 1, 0])
y_pred = np.array([1, 0, 1, 0, 0, 1, 1, 0, 1, 0])

# Compute all metrics
print("Accuracy :", accuracy_score(y_true, y_pred))
print("Precision:", precision_score(y_true, y_pred))
print("Recall   :", recall_score(y_true, y_pred))
print("F1 Score :", f1_score(y_true, y_pred))

# Confusion Matrix
print("\nConfusion Matrix:")
print(confusion_matrix(y_true, y_pred))

# Detailed report
print("\nClassification Report:")
print(classification_report(y_true, y_pred))

📊 Regression Metrics

from sklearn.metrics import (
    mean_absolute_error, mean_squared_error, r2_score
)
import numpy as np

y_true = np.array([3.0, 5.0, 2.5, 7.0, 4.5])
y_pred = np.array([2.8, 5.2, 2.1, 6.8, 4.9])

print("MAE :", mean_absolute_error(y_true, y_pred))
print("MSE :", mean_squared_error(y_true, y_pred))
print("RMSE:", np.sqrt(mean_squared_error(y_true, y_pred)))
print("R²  :", r2_score(y_true, y_pred))